A Navy neurologist, Capt. Elwood Hopkins, has posted a 3-part article on “The Power of Acupuncture” on Navy Medicine Live, the official blog of Navy and Marine Corps Health Care. It can serve as a useful lesson in how not to think about medicine. It is a prime example of how an intelligent, educated doctor can be fooled and can fool himself into thinking that a placebo is an effective treatment.
To set the scene: acupuncture has been increasingly accepted in military circles. The Air Force is teaching its doctors “battlefield acupuncture” based on the faulty evidence of one Air Force doctor, Richard Niemtzow. The Army is using it to treat PTSD. The Navy offers it too.
Hopkins says that after 40 years of practicing neurology, “It was only natural to begin thinking about something else.” (Why? Boredom? And why pick acupuncture?) When he got an e-mail from his Specialty Leader announcing the opportunity for Navy doctors to learn how to do acupuncture, he submitted his application that same day. He was undoubtedly impressed that this training was being offered by the Navy, lending it the imprimatur of authority. His prior impression of acupuncture was that it was a “mysterious tool” that seemed to work; and instead of asking critical questions, he says he was looking for “a fundamental scientific understanding of acupuncture” and asking to see the supporting research and data.
Mistake #1: Prior prejudice. It sounds like he already had a favorable opinion of acupuncture and was predisposed to accept it.
Mistake #2: Confirmation bias. He was looking only for confirmation rather than also looking for any disconfirming research.
He says his intellect was actively engaged by the teacher, a “charismatic master acupuncturist,” who laid a neurophysiologic foundation for how acupuncture works (never mind showing that it does work!).
Mistake #3: Getting information from a questionable source. A “master” acupuncturist is hardly likely to present a balanced picture of the evidence for and against his source of livelihood. Hopkins himself calls the teacher “charismatic,” so he could have suspected that he might be influenced by that charisma to accept things he would not have accepted as readily from a dry, objective presentation of scientific evidence.
He was told about local physiologic changes in tissues stimulated by needles. He was not told that non-needle “acupuncture” (with electrical stimulation through intact skin or with simple touching with toothpicks) had been shown equally effective, even when acupuncture points are avoided. He was told that needles caused reversal of tissue acidosis. (This is a claim I don’t remember hearing before, and I think it is based on a couple of Chinese studies in animals. Even if true, its clinical relevance would be questionable.) He was told about the “gate control” hypothesis, but was not told that after half a century of investigation it has not been accepted as the explanation for acupuncture’s effects. He was told about MRI findings and endorphin release, but was not told that the same findings can be elicited by placebo pills. I see them as evidence of the mechanism for acupuncture’s placebo effects; he interprets them as evidence that acupuncture “resets normal controls within the autonomic nervous system and maintains CNS homeostasis.” What does that even mean? Sounds to me like typical CAM pseudoscientific doublespeak.
Mistake #4: Cherry-picking the literature. The charismatic master acupuncturist snowed Hopkins with every shred of data that might possibly support a physiologic mechanism for acupuncture, even providing “an extensive reference library.” Did he disclose that it doesn’t matter where you put the needles? Did he list the high-quality trials showing that sham acupuncture works just as well? Did he list all the negative systematic reviews or Edzard Ernst’s recent systematic review of systematic reviews of acupuncture for pain? It is obvious that he cherry-picked the literature to support his claims. This is easy to do with acupuncture. Many low-quality studies of acupuncture are available.
In part 2 Hopkins actually asks if the clinical effects might be due to placebo. He wonders how we would know, since there is “no honest way” to do a properly controlled double blind study. His teacher says it is better to go by the functional outcome rather than by patient reports of pain levels. So far, so good. But then Hopkins throws science out the window and never mentions placebo again. He actually says
There is nothing like personal experience to convince one of an effect. It is a bit like not requiring a double blind placebo controlled cross-over study to establish that an open parachute is more effective than a closed one.
Wow! Unbelievable! This sounds like it was written by someone ignorant of science and logic rather than by a neurologist. His analogy is a clichéd fallacy: we don’t accept the effectiveness of parachutes because we have had personal experience jumping out of planes. I think he meant to say that not every claim requires proof by placebo-controlled trials, which is true but not pertinent here. We don’t need to do controlled trials to find out if it is effective to do appendectomies for appendicitis or to set broken bones, but we do require controlled trials to find out if acupuncture works. In a way it’s true that personal experience is the best way to convince someone there is an effect, but it’s useless for determining whether there really is an effect. To rephrase his statement, there is nothing like solid scientific data to convince a scientist who knows better than to accept personal experience as evidence.
Mistake #5: Not understanding why science is necessary. It’s hard to believe that this was written by someone who has gone through medical school and residency training. It’s a sad indictment of our educational system.
The instructor asks for volunteers and Hopkins offers himself as a guinea pig. The instructor treats him for his Raynaud’s disease, telling him he believes that it is due to prior cervical injury. As a neurologist, Hopkins should know that the term “Raynaud’s disease” refers only to idiopathic cases and if the condition is secondary to some instigating factor, it is called “Raynaud’s syndrome.” Also, while repetitive trauma from vibrating tools like jackhammers and prior injuries to the hands or feet have been recognized as causes of Raynaud’s, “cervical injury” has not. There are studies showing that acupuncture is more effective for Raynaud’s than drugs or than no treatment, but they are not convincing because they didn’t use placebo control groups. After the treatment, Hopkins’ symptoms resolved, and he became a believer.
Mistake #6: Relying on his personal experience. True believers ask us to “try it yourself” and they say “I saw it with my own eyes.” We have ample evidence that seeing something with our own eyes is often misleading, and that trying something for yourself can interfere with your ability to objectively assess the evidence.
Mistake #7: The post hoc ergo propter hoc fallacy. Hopkins assumes that because his symptoms improved after the treatment, they improved because of the treatment. He doesn’t even consider that there might be other explanations or confounding factors. (For one thing, emotional stress is a known trigger for Raynaud’s symptoms.)
His classmates were treated for various conditions including radiculopathies and interstitial cystitis, and they all “benefitted.” He calls acupuncture a “safe and inexpensive tool that has been time-tested for several thousand years.”
Mistake #8: Relying on the personal experience of others. Testimonials abound for even the quackiest of quack treatments. No matter how many anecdotes we manage to accumulate, the plural of anecdote is not data: that’s why we do science.
Mistake #9: The ancient wisdom fallacy. He is dead wrong about acupuncture being several thousand years old. Even if it was, length of use is no indication of truth. Astrology has been around for longer than acupuncture: does he think it’s valid because it has been time-tested? Blood-letting was “time-tested” for many centuries, but it turned out to do more harm than good.
In Part 3, Hopkins relates how he has implemented acupuncture in his practice. He has given over 1000 acupuncture treatments for everything from headaches to prostatitis. (One wonders why a neurologist would be treating prostatitis.) He claims a 90% success rate with many spectacular responses. He actually uses the word “miracle.” He admits that some patients don’t respond, saying it is “never clear why.” (I think I can guess why!) He reports improvements in control of diabetes and hypertension, less need for medication, better sleep, etc. He concludes that “It is now evident to me that there truly is a great benefit to acupuncture.” He says it is safe (although the Ernst study documented rare but serious adverse effects including death). He says there are no contraindications, but numerous lists of contraindications are easily found on the Internet both on acupuncture websites and on mainstream medical websites. He says the only time he would not use it was if the patient didn’t want it. He recommends that anyone caring for patients should consider adding this tool to their kit.
Mistake #10: Making uncontrolled observations. His patients improved, but how many of them would have improved without any treatment or with a credible placebo offering some of the non-specific treatment effects of acupuncture?
Mistake #11: Proselytizing on the basis of his own uncontrolled observations. Now that he has convinced himself, he wants to persuade others by simple assertions and by the same kind of unreliable “evidence” that convinced him.
Mistake #12: Not doing his own research. He might have checked PubMed. He might have read a neat new study confirming previous evidence that acupuncture is no more effective than placebo. It showed that patients were more likely to improve if they believed in acupuncture and believed they got the real thing rather than a placebo, regardless of which they actually got. He might have read what Yale neurologist Steven Novella wrote after actually researching the literature on acupuncture himself. He might have read the many negative systematic reviews, such as the one showing that a “small analgesic effect of acupuncture was found, which seems to lack clinical relevance and cannot be clearly distinguished from bias” or the recent systematic review of systematic reviews by Edzard Ernst showing “numerous contradictions and caveats.” He might have read the many skeptical articles on science blogs. He might have read The Skeptic’s Dictionary entry on acupuncture. He might have consulted Quackwatch’s affiliate Acupuncture Watch. Even reading the acupuncture article on Wikipedia might have raised some doubts in his mind.
At this point, even if he is willing to look at the great mass of disconfirming evidence, he is probably not capable of judging it objectively. Once someone has become a true believer on the basis of personal experience reinforced by grateful patient feedback, there is rarely any hope. He has drunk the Kool-Aid. I just hope he doesn’t go on to seek training from a charismatic homeopath or a reiki master!
I can understand why many doctors are less skeptical than they should be about most of the CAM information they encounter: they are used to having pre-digested, accurate scientific information presented to them by experts. They were (sadly) not taught to question what their teachers said in medical school and residency. They were taught about evidence-based medicine, but they have poor understanding of what we mean by science-based medicine. They rely on published information in medical journals, but they may not realize that half of the studies they read are wrong. Even if they are good at critically evaluating scientific medical information, they may not be used to critically analyzing information from the realm of CAM. They may not have learned to recognize the common logical fallacies and the pitfalls.
The One Big Mistake: Not Following the SkepDoc’s Rule of Thumb. My rule, which applies to critical thinking in every sphere: before accepting any new claim, find out who disagrees with it and why. Once you fully understand the arguments on both sides, only then are you qualified to judge whether the claim is credible. What if a jury listened to the prosecution but not to the defense?
For those of us who have made an effort to develop our critical thinking skills and have not been influenced by personal experience or charismatic apologists, all indications are that acupuncture amounts to an elaborate placebo system. Using placebos on patients is unethical. As a retired Air Force colonel and as a physician, I am saddened to see acupuncture infiltrate the military health care system. And I am saddened to see how Dr. Hopkins’ faulty thinking led him astray. To salvage a bit of good from the bad, I hope his experience can serve as a bad example for others to learn from.